German Compound Analysis with wfsc

Compounding is a very productive process in German to form complex nouns and adjectives which represent about 7% of the words of a newspaper text. Unlike English, German compounds do not contain spaces or other word boundaries, and the automatic analysis is often ambiguous. A (non-weighted) finite-state morphological analyzer provides all potential segmentations for a compound without any filtering or prioritization of the results. The paper presents an experiment in analyzing German compounds with the Xerox Weighted Finite-State Compiler (wfsc). The model is based on weights for compound segments and gives priority (a) to compounds with the minimal number of segments and (b) to compound segments with the highest frequency in a training list. The results with this rather simple model will show the advantage of using weighted finite-state transducers over simple FSTs. 1 Compound Construction A very productive word formation process in German is compounding, which combines words to build more complex words, mainly nouns or adjectives. In a large newspaper corpus the Xerox German Morphological Analyzer [1] identified 5.5% of 9,3 million tokens and 43% of overall 420,000 types as compounds. In other texts, such as technical manuals, the percentage of compound tokens may even increase (e.g. 12% in a short printer manual). This is comparable to the observations of Boroni et al. [2] who found in a 28 million newswire corpus that 7% of the tokens and 46% of the types were compounds. Regarding the construction of compounds, any adjective or noun (including proper names) may, in principle, appear as head word, and any 1 Tokens represent all words of the text, types count only different word forms. 2 Verbal compounds (such as spazierengehen) exist, but are much less productive than nouns or adjectives. They are not taken into account in this experiment. adjective, noun or verb may occur as the left-hand (“modifier”) part of a compound. – Buchseite (book page) – Großstadt (big town) – grasgrün (grass green) – Goethestück (Goethe piece) The Xerox finite-state tool [3] for German morphological analysis [1] implements this general principle without any semantic restrictions and with only a few morphosyntactic constraints concerning the so-called “linking” elements. The advantage of this very general approach is high and robust coverage. The inconvenience is potentially very ambiguous output due to “over-segmentation” of (long) words. A potential source of over-segmentation is the homonymy of compound parts with derivational affixes. – derivational suffix -ei (ing) vs. noun Ei (egg), e.g. Spielerei (playing) – Vogel#ei (bird egg) – prefix ein(in-) vs. cardinal ein (one), e.g. Einwohner#zahl (inhabitant number) – Ein#zimmer#wohnung (one room